Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition

Rapid progress and superior performance have been achieved for skeleton-based action recognition recently. In this article, we investigate problem under a cross-dataset setting, which is new, pragmatic, challenging task in real-world scenarios. Following the unsupervised domain adaptation (UDA) paradigm, labels are only available on source dataset, but unavailable target dataset training stage. Different from conventional adversarial learning-based approaches UDA, utilize self-supervision scheme to reduce shift between two datasets. Our inspiration drawn Cubism, an art genre early 20th century, breaks reassembles objects convey greater context. By segmenting permuting temporal segments or human body parts, design self-supervised learning classification tasks explore spatial dependency of improve generalization ability model. We conduct experiments six datasets recognition, including three large-scale (NTU RGB+D, PKU-MMD, Kinetics) where new settings benchmarks established. Extensive results demonstrate that our method outperforms state-of-the-art approaches. The codes model all compared methods at https://github.com/shanice-l/st-cubism.